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Abstract: We consider a Markovian clearing queueing system, where the customers are accu- 
mulated according to a Poisson arrival process and the server removes all present customers at 
the completion epochs of exponential service cycles. This system may represent the visits of a 
transportation facility with unlimited capacity at a certain station. The system evolves in an 
alternating environment that influences the arrival and the service rates. We assume that the 
arriving customers decide whether to join the system or balk, based on a natural linear reward- 
cost structure. We study the balking behavior of the customers and derive the corresponding 
Nash equilibrium strategies under various levels of information. 

Keywords: Queueing, Stochastic clearing system. Alternating random environment. Balking, 
Nash equilibrium strategies 



1 Introduction 

Queueing systems with batch services are often used to represent the visits of a transportation 
facility at a certain station. This allows the quantification of the congestion of the station and 
can be used to take control measures (e.g. changing the frequency of the visits), so that the 
quality of service is kept within acceptable limits. The capacity of the facility is usually assumed 
unlimited. This is justified, because in most applications the capacity of the facility is chosen 
large enough, so that the probability that some waiting customers cannot be accommodated is 
negligibly small. Moreover, the waiting customers that cannot be served at a visit of the facility 
are not in general willing to wait for its next visit and abandon the system. Therefore, it is 
realistic to assume that all present customers are removed at the visit points of the facility. Such 
systems are referred to as stochastic clearing systems. 



Stochastic clearing systems have been studied extensively in the literature (see e.g. Stid- 
ham (1974), Serfozo and Stidham (1978), Artalejo and Gomez-Corral (1998) and Yang, Kim 
and Chae (2002)). They have been also studied in the framework of stochastic systems subject 
to (total) catastrophes or disasters, where catastrophic events are assumed to remove all the 
customers/units of the system/population (see e.g. Kyriakidis (1994), Economou and Fakinos 
(2003,2008), Stirzaker (2006,2007) and Gani and Swift (2007)). In the majority of such studies 
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the interest of the investigators lies in the transient and/or the stationary distribution of the 
process of interest. However, optimization issues for this class of systems have also attracted 
the interest in the literature (see e.g. Kyriakidis (1999a,b), Economou (2003), Kyriakidis and 
Dimitrakos (2005)). 

During the last decades, there is an emerging tendency to study queueing systems from an 
economic viewpoint. In the context of stochastic clearing systems, the optimization questions 
that have been considered so far concern the central planning of the systems. In these studies, 
the objective is the determination of optimal policies for the server, about when he should re- 
move the customers from the system (see e.g. Stidham (1977), Kim and Sella (1993), Economou 
(2003), Kyriakidis and Dimitrakos (2005)). However, to the best of our knowledge, there are 
no economic studies that concern the behavior of the customers when they are free to make 
decisions to maximize their own benefit. Such considerations lead to a game theoretic economic 
analysis of their behavior in the system. 

In general, the economic analysis of customer behavior in a queueing system is based on 
some reward-cost structure which is imposed on the system and reflects the customers' desire 
for service and their unwillingness to wait. Customers are allowed to make decisions about their 
actions in the system, for example they may decide whether to join or balk, to wait or abandon, 
to retry or not etc. The customers want to maximize their benefit, taking into account that the 
other customers have the same objective, and so the situation can be considered as a game among 
them. In this type of studies, the main goal is to find individual and social optimal strategies. 
The study of queueing systems under a game-theoretic perspective was initiated by Naor (1969) 
who studied the M/M/1 model with a linear reward-cost structure. Naor (1969) assumed that 
an arriving customer observes the number of customers and then makes his decision whether to 
join or balk (observable case). His study was complemented by Edelson and Hildebrand (1975) 
who considered the same queueing system but assumed that the customers make their decisions 
without being informed about the state of the system. Since then, there is a growing number of 
papers that deal with the economic analysis of the balking behavior of customers in variants of 
the M/M/1 queue, see e.g. Hassin and Haviv (1997) (Af/M/1 queue with priorities), Burnetas 
and Economou (2007) (M/M/1 queue with setup times), Guo and Zipkin (2007) (A/ /M/1 queue 
with various levels of information and non-linear reward-cost structure), Hassin (2007) {M/M/\ 
queue with various levels of information and uncertainty in the system parameters), Economou 
and Kanta (2008a, b) (M/M/1 queue with compartmented waiting space, M/M/1 queue with 
unreliable server). Sun, Guo and Tian (2010) (M/M/1 queue with setup/closedown times) and 
Zhang and Wang (2010) (M/M/1 queue with delayed repairs). The monographs of Hassin and 
Haviv (2003) and Stidham (2009) summarize the main approaches and several results in the 
broader area of the economic analysis of queueing systems. 

The aim of the present paper is to study the equilibrium behavior of the customers regard- 
ing balking in the framework of a Markovian clearing queueing model. The balking behavior 
of customers in stochastic clearing systems that model transportation stations is important and 
should be taken into account if one wants to obtain a reliable representation of what is going 
on in these systems. However, such systems usually evolve in random environment, i.e. there is 
some external process that influences the arrival and the service rates. In the present study we 
will concentrate on a clearing system evolving in an alternating random environment (modeled 
by a 2-state continuous-time Markov chain). We will determine equilibrium balking strategies 
for the customers under various levels of information. In particular, we will consider several 
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information cases, as an arriving customer may observe or not the number of customers in the 
system and/or the state of the environment, before making his decision about whether to join 
or balk. 

The paper is organized as fohows. In Section 2, we describe the stochastic dynamics of the 
model, the reward-cost structure and the decision framework (information cases). In Section 3 
we consider those cases, where the strategies of the other customers do not influence the ex- 
pected net benefit of a tagged customer. These are the unobservable cases, where the tagged 
customer does not observe the number of customers in the system before making his decision 
and the fully observable case, where he observes both the number of customers in the system 
and the state of the environment. In all these cases, we show that the expected net benefit of 
a tagged customer depends only on his strategy and not on the strategies followed by the other 
customers, a fact that implies the existence of dominant strategies. This is a special feature of 
the system that is related to the nature of the stochastic clearing mechanism. In Sections 4 and 
5 we consider the almost observable case, where the customers get informed upon arrival about 
the number of waiting customers in the station but not about the state of the environment. In 
this case, the waiting customers do not imply any additional cost on the individual, but their 
presence provides a signal about the clearing rate. Depending on the parameters of the model, a 
large number of waiting customers may increase or decrease the conditional probability that the 
clearing rate is the slow one. In Section 4, we present some preliminary results. More concretely, 
we first compute the stationary distributions of the system, when the customers follow either 
a threshold or a reverse-threshold strategy. Then we compute the net benefit of an arriving 
customer who decides to join, given that he observes n customers and that the others follow 
a threshold or a reverse-threshold strategy. In Section 5, we conclude our study and we char- 
acterize all equilibrium strategies within the class of threshold and reverse-threshold strategies. 
The main contribution of the paper is an algorithm that computes efficiently all these equilib- 
rium strategies. In Section 6, we summarize our findings and discuss the Follow-The-Crowd and 
Avoid-The-Crowd notions for this model as well as the problem of social optimization. 



2 The model 

We consider a transportation station with infinite waiting space that operates in an alternating 
environment. The environment is specified by a 2-state continuous-time Markov chain {E{t)}, 
with state space = {1,2} and transition rates qeg/, for e 7^ e'. Whenever the environment is 
at state e, customers arrive according to a Poisson process at rate Ae, whereas a transportation 
facility visits the station according to a Poisson process at rate fie- The two Poisson processes 
are assumed independent. At the visit epochs of the transportation facility all customers are 
served instantaneously and removed from the station. Therefore, we have a stochastic clearing 
system in an alternating random environment. 

We represent the state of the station at time i by a pair (N{t), E{t)), where N{t) records 
the number of customers at the station and E{t) denotes the environmental state. The stochas- 
tic process {{N{t), E[t)) : t > 0} is a continuous-time Markov chain with state space S^'^ = 
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{(n, e) : n > 0, e 



1, 2} and its non-zero transition rates are given by 



Q(n,e){n+l,e) 



Q{n,e){Q,e) 
9{n,l){n,2) 
Q{n,2){n,l) 



Ae, n > 0, e = 1, 2, 
fie, n>l, e = 1, 2 
912, ?^ > 0, 
921, n > 0. 



(2.1) 
(2.2) 
(2.3) 
(2.4) 



We define pe = — , e = 1, 2. The value of pe can be thought of as a measure of congestion of the 
system under the environmental state e, as it expresses the mean number of customers accu- 
mulated between two successive visits of the transportation facility (given that the environment 
remains continuously in state e). 

We are interested in the behavior of customers, when they have the option to decide whether 
to join or balk. We assume that a customer receives a reward of R utility units for completing 
service. Moreover, a customer accumulates costs at a rate of C utility units per time unit that 
he remains in the system. We also assume that customers are risk neutral and wish to maximize 
their net benefit. Finally, their decisions are assumed irrevocable, in the sense that neither 
reneging of entering customers nor retrials of balking customers are allowed. 

Since all customers are assumed indistinguishable, we can consider the situation as a sym- 
metric game among them. Denote the common set of strategies (set of available actions) and 
the utility (payoff) function by S and U respectively. More concretely, let U {stagged-, Sothers) be 
the payoff of a tagged customer who follows strategy stagged, when all other customers follow 
Sothers- A strategy si is said to dominate strategy S2 if (si, s) >U (s2, s), for every s E 5. A 
strategy s^, is said to be dominant if it dominates all other strategies in 5. A strategy s is said 
to be a best response against a strategy Sothers, if ^ (s, •Sothers) > ^ {stagged, s others), for every 
Stagged £ <S ■ Finally, a strategy Sg is said to be a (symmetric) Nash equilibrium, if and only 
if it is a best response against itself, i.e. U (se,Se) > U {s,Se), for every s £ S. The intuitive 
interpretation of a Nash equilibrium is that it is a stable point of the game, in the sense that if 
all customers agree to follow it, then no one can benefit by deviating from it. We remark that 
the notion of a dominant strategy is stronger than the notion of an equilibrium. In fact, every 
dominant strategy is an equilibrium, but the converse is not true. Moreover, while equilibrium 
strategies exist in most situations, dominant strategies rarely do. 

In the next sections we obtain customer equilibrium strategies for joining/balking. We 
distinguish four cases depending on the information available to the customers at their arrival 
instants, before the decision is made: 

• Fully unobservable case: Customers do not observe N{t) nor E{t). 

• Almost unobservable case: Customers do not observe N{t), but observe E{t). 

• Fully observable case: Customers observe both N{t) and E(t). 

• Almost observable case: Customers observe N{t), but do not observe E{t). 

From a methodological point of view, the first three cases are similar and they lead to 
dominant strategies, so we study all of them in Section [31 The almost observable case which is 
the most interesting and methodologically demanding is treated in Sections [H and O 
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3 The unobservable and the fully observable cases: Dominant 
strategies 



Let Se denote the time till the next arrival of the transportation facility, given that the envi- 
ronment is at state e. A moment of reflection shows that S^. is independent of the number of 
customers in the system, because of the mechanism of the total removals of customers at the 
visits of the facility and the memoryless property of the exponential distribution. By employing 
a first-step argument, conditioning on the next transition of the Markov chain {{N{t), E{t))} 
that is either a visit of the facility or a change in the environment, we obtain the equations 

E[S,] = -^— + ^^.0 + ^l^.E[S2], (3.1) 

/Ui + qi2 fj-i + qi2 fj-i + qi2 

E[S2] = -^— + ^^-0 + ^^-E[Si]. (3.2) 



E[S,] = + + , (3.3) 

/^iM2 + fj-mi + iJ'2qi2 

E[S,] = + + . (3.4) 

/^lAi2 + /^lQ'21 + Ai2Q'l2 



Solving ([31])- ((321) yields 



3.1 The fully unobservable case 

We can now proceed and determine the equilibrium strategies of the customers in the fully 
unobservable case. A general balking strategy in the fully unobservable case is specified by a 
single joining probability q. The case g = corresponds to the pure strategy 'to balk' whereas 
the case (/ = 1 corresponds to the pure strategy 'to join'. Any value of (7 G (0, 1) corresponds to 
a mixed (randomized) strategy 'to join with probability q or balk with probability 1 — q\ We 
have the following Theorem 13.11 

Theorem 3.1 In the fully unobservable model of the stochastic clearing system in alternating 
environment, there always exists a dominant strategy. The dominant strategies depend on the 
relative value of the ratio ^ with respect to the critical value 

Aig2i/^2 + A2gi2/^i , 521 + qi2 ,^ 

Vfu = TT ^ TT T H • (3.5) 

{Mq21 + Mqi2){fJ'llJ-2 + Ail'?21 + fJ'2qi2) WAt2 + Aiig'21 + /^2gi2 

We have three cases that are summarized in TableUl 



Value of ^ 


U < 


R T/ 




Dominant strategy(ies) 





q G [0, 1] 


1 



Table 1: Dominant strategies in the fully unobservable case 
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Proof. Suppose that the customers follow a certain strategy and consider a tagged customer 
upon arrival. The probability that he finds the environment at state e is 

{XiPe[1) + \2Pe{^)) 

where {pE{e),e = 1,2) is the stationary distribution of the environment which is given by 

Pe{1) = -^f- (3.7) 

921 + 912 

Pe{2) = -J^. (3.8) 

921 + 912 

Therefore, the expected net benefit of the tagged customer if he decides to join is given by 

2 

Sfu = '^PE-arrival{e){R - CE[Se\), (3.9) 
e=l 

where E[Se\ are given by ([331) -([331). Plugging ([STll-dSS]) in (^M and substituting in ([33]) yields 

Xiq2iE[Si] + A29i2-E'[5'2] 



Sfu — R — C 



Ai92i + A2912 



= R-c(- ^ + ^^^^^^^ , + '-2^±^ V(3.10) 

V(Al921 + A29l2)(/WlAi2 + /W192I + fJ'2qi2) fJ'lfJ'2 + Ml921 + Ai29l2/ 

The tagged customer prefers to join if Sju > 0, prefers to balk if Sfu < and he is indifferent 
between joining and balking if Sfu = . Solving with respect to ^, we obtain the three cases 
of Tabled! ■ 



3.2 The almost unobservable case 

We can now proceed and determine the equilibrium strategies of the customers in the almost 
unobservable case. A general balking strategy in the almost unobservable case is specified by 
an ordered pair of joining probabilities (91, 92), where qe is the joining probability of a customer 
if the environmental state upon arrival is e, e = 1,2. We have the following Theorem 13.21 



Theorem 3.2 In the almost unobservable model of the stochastic clearing system in alternating 
environment, there always exists a dominant strategy. The dominant strategies depend on the 
relative value of the ratio ^ with respect to the critical values 

ymin ^ min(^i,/i2) + 921 +912 ymax ^ max(^i , ^2) + 921 + 9l2 
fJ-ifJ-2 + Aii92i + Ai29i2 ' /Ui/^2 + Aii92i + /U2912 

// ^1 7^ /U2 , then V^"" < Km"^ and we have five cases that are summarized in Table [H 



6 



Value of § 


R ^ '\rmin 
C ^ ^ au 


R \rmin 
C ~ ^ au 


^ au C ^ au 


C ~ ^au 




Dominant 
strategy (ies), 
when /J-i < 1^2 


(0,0) 


(0,^2), 

92 G [0, 1] 


(0,1) 


('?1,1), 

qi G [0, 1] 


(1,1) 


Dominant 
strategy (ies), 
when /.ii > /.i2 


(0,0) 


(91,0), 
91 G [0, 1] 


(1,0) 


(1,92), 

92 G [0, 1] 


(1,1) 



Table 2: Dominant strategies in the almost unobservable case, when /ii / //2 



// /ii = /i2, ^^en ^4™" = V^™^. i^ei denote the common value ofV^''"' and . We 
have three cases that are summarized in Tablel^ 



Value of ^ 




R-V 


Q ^ ^ au 


Dominant strategy (ies), 
when fii = fi2 


(0,0) 


(91,92), 

91,92 G [0,1] 


(1,1) 



Table 3: Dominant strategies in the almost unobservable case, when ^1 = ^2 

Proof. Consider a tagged customer that observes the state of the environment upon arrival. If 
he decides to join given that he finds the environment at state e, then his expected net benefit 
will be 

Sau{e) = R-CE[Sel (3.12) 

where E[Se\ are given by (j3.3p - ()3.4p . The customer prefers to join if Sau{e) > 0, which is written 
equivalently as ^ > E[Se]- Similarly, we have that he prefers to balk if ^ < E[Se] and he is 
indifferent between joining and balking if ^ = £^[S'e]. By considering the various possible cases 
with regard to the order of the three quantities ^, E[Si] and -£^[52], we obtain the corresponding 
cases in the statement of the Theorem 13. 2i Note that the strategies prescribed in the Theorem 
13.21 are dominant, since they do not depend on what the other customers do, i.e. they are best 
responses against any strategy of the others. ■ 

3.3 The fully observable case 

Regarding the fully observable case, where the arriving customers observe both the number of 
waiting customers and the state of the environment, the situation is identical to the almost 
unobservable case. This happens because the mean sojourn time of an arriving customer, given 
that he finds n customers in the system and the environment at state e does not depend on n. 
Therefore, if the environmental state e is observed upon arrival, then the information about the 
number of customers n is superfluous and is discarded by the customers. We conclude that the 
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dominant balking strategies are the ones described in Theorem 13.21 



4 The almost observable case: Preliminaries 

In this section, we consider the almost observable case. In this case, the customers, upon ar- 
rival and before making their decisions about whether to join or balk, observe the number of 
customers in the system but not the state of the environment. Thus a general balking strategy 
in this case is specified by a vector of joining probabilities {Oq, 9i, 62, ■ ■ ■), where 6i is the joining 
probability of a customer that sees i customers in the system upon arrival (excluding himself). 

Suppose that a tagged customer observes n customers in the system upon arrival. Although 
his mean sojourn time does not depend on n, the information about n influences the probabilities 
that the environment is found at state 1 or 2. We expect intuitively that there are two cases: 
Either the 'slow service' environmental state e with = miii(/Ui, /i2) coincides to the 'more 
congested' environmental state e' with pe' = max{pi, P2) or it coincides to the 'less congested' 
environmental state e" with p^i' = min(pi,/92). In the former case, the greater the number n 
of the customers found by the tagged customer, the more probable is that the environment is 
found at the 'slow service' environmental state. Therefore, the tagged customer becomes less 
willing to join the system as n increases. Thus, we expect that the tagged customer will benefit 
from joining the system, if the number of customers n is below a certain threshold, i.e. he will 
adopt a threshold strategy. On the contrary, in the latter case, the situation is reversed. Then, 
the greater the number n of the customers found by a tagged customer, the more probable is 
that the environment is found at the 'fast service' environmental state. Therefore, we expect 
that the tagged customer will benefit from joining the system, if the number of customers n 
exceeds a certain threshold, i.e. he will adopt a so called reverse-threshold strategy. Following 
this reasoning, we will limit our search for equilibrium strategies within the class of threshold 
and reverse-threshold strategies. As we will see, this family is rich enough to ensure the existence 
of an equilibrium strategy for any values of the underlying parameters of the model. 

Definition 4.1 A balking strategy {9o,Oi,92, ■ ■ ■), where 9i is the joining probability of a cus- 
tomer that sees i customers in the system upon arrival (excluding himself) is said to be a mixed 
threshold strategy, if there exist uq G {0, 1, . . .} and 9 € [0, 1] such that 9i = 1, for i < uq, 9ng = 9 
and 9i = 0, for i > uq. Such a strategy will be referred to as the {no,9)-mixed threshold strategy 
(symbolically the [no, 6*] strategy) and it prescribes to join if you see less than uq customers, 
to join with probability 9 if you see exactly no customers and to balk if you see more than uq 
customers. 

An {no,0)-mixed threshold strategy which prescribes to join if you see less than no customers 
and to balk otherwise will be referred to as the n^-pure threshold strategy (symbolically the [no] 
strategy). 

A balking strategy {9q, 9i, 92, . . .) is said to be a mixed reverse-threshold strategy, if there exist 
no G {0, 1, . . .} and 9 G [0, 1] .such that 9i = 0, for i < no, 9no = 9 and 9i = 1, for i > uq. Such a 
strategy will be referred to as the (no, 9)-mixed reverse-threshold strategy (symbolically the [no, 
strategy) and it prescribes to balk if you see less than no customers, to join with probability 9 if 
you see exactly no customers and to join if you see more that hq customers. 

An {nQ,l)-mixed reverse-threshold strategy which prescribes to join if you see at least tiq 
customers and to balk otherwise will be referred to as the n^-pure reverse-threshold strategy 
(symbolically the [no J strategy). 
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The strategy which prescribes to join in any case is considered to be both a threshold and 
a reverse-threshold strategy (symbolically the [oo] or [Oj strategy). The same is true for the 
strategy which prescribes to balk in any case (symbolically the [0] or [ooj strategy). 



4.1 Stationary distributions 

In this subsection, we determine the stationary distributions of the system, when the customers 
follow any given strategy from the ones that have been described in Definition 14. II We will first 
determine the stationary distribution of the original system when all customers join. The result 
is reported in the following Proposition 14.11 

Proposition 4.1 Consider the stochastic clearing system in alternating environment, where all 
customers join. The stationary distribution {p{n,e)) is given by the formulas 

= + „>0. (4.1) 



where 



' ^/A(l -zi) 

^ ^ (/ilA2Z2 + 1^1112 + At2gl2 + fliq2l)PE{l) 
' VA{1 - Z2) ' 

. ^ (a^2Ai^i + /ii/i2 + A^2gi2 + /iig2i)pi;(2) ,^ . 

(/X2A1Z2 + /xi^2 + Ai2gi2 + Mi92i)p£;(2) , . 

VA(1 - Z2) 

A = [A2(Aii + g'12) - Ai(/X2 + 921)]^ + 4AiA2gi2g2i, (4.7) 
-Ai(/i2 + 921) - A2(^i + 912) ± \/A 
"^'^ = (^-^^ 

and pe{^), Pe{'^) o,re the stationary probabilities of {E{t)} given from ()3.7p - p.8p . 

Proof. For the stationary analysis, note that the state of the system is described by a continuous- 
time Markov chain with state space 

gN,E ^ i(^^g^) . ^ > g ^ with its non-zero transi- 

tion rates given by ([2?l])- dm]). The corresponding stationary distribution (p(n, e) : (n, e) G 5^'^) 
is obtained as the unique positive normalized solution of the following system of balance equa- 
tions: 



(Ai + + gi2M0, 1) = g2ip(0,2) + J^^ip(n,l), (4.9) 

n=0 

(Ai+^i + gi2)p(n,l) = g2ip(n,2) + Aip(n-l,l), n > 1, (4.10) 

00 

(A2 + /i2 + g2i)p(0,2) = (7i2p(0,l) + J^^2P(n,2), (4.11) 

n=0 

(A2 + ^2 + g2i)p(n,2) = gi2p(n,l) + A2p(n-l,2), n > 1, (4.12) 
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where we have included in (|4.9|) . (|4.1ip the pseudo-transitions from (0,e) to (0,e), e = 1,2, 
with rate /Ug, that correspond to visits of the facility at an empty system. Note also that the 
underlying Markov chain is always positive recurrent as the stochastic clearing mechanism en- 
sures that starting from the state (0, 1), the process will visit it again with probability 1 and 
the corresponding mean time is finite. 

For determining the stationary probabilities, we may follow the standard probability gener- 
ating function approach. Thus, we define the partial stationary probability generating functions 
of the system as 

oo 

Ge{z) = ^p{n,e)z'^, \z\ < 1, e = l,2. (4.13) 

n=0 

and we observe that Gi(l) = pe{1), G2(1) = pe{2) with psil), PeC^) given from (13771) - ([3:81) . 
Summing equation ()4.9p and equations (I4.10p multiplied hy z"', n > I, yields after some straight- 
forward algebra a linear equation in Gi{z) and G2{z). Similarly, equations (|4.1ip and (|4.12p . 
n > 1, yield another linear equation in Gi{z) and G2{z). Solving the system of these equations 
we obtain Gi{z) and 6*2(2) as rational functions of z with known coefficients expressed in terms 
of the parameters of the model. Using partial fraction expansion and then expanding the simple 
fractions in powers of z yields (|4.ip and (j4.2p . Indeed, by direct substitution, we can easily check 
that p(n, 1) and p{n, 2) given by (14. ip and (14. 2p satisfy (14.10p . By a simple summation, we can 
also check that p{n, 1) and p{n, 2) given by ()4.ip and ()4.2p satisfy (|4.9p . The validity of (|4.12p 
and (|4.1ip is checked similarly. ■ 

We will now deduce the stationary distribution of the system when the customers follow a 
mixed threshold strategy. We have the following Proposition 14.21 

Proposition 4.2 Consider the almost observable model of the stochastic clearing system in 
alternating environment, where the customers join the system according to the {no,9)-mixed 
threshold strategy. The corresponding stationary distribution {pao{n,e; \nQ,6~\)) is given by the 
formulas 

Pao{n,e;\no,e]) = p{n,e), < n < no - 1, e = l,2, (4.14) 

00 

Paoino,e;\no,9]) = (1 - e)"-X7i, e), e = l,2, (4.15) 

n=no 
00 

Pao{no + l,e;\no,9]) = ^ [1 - (1 - 0)"-"Xn, e), e = l,2, (4.16) 

n=no+l 

Paoin,e;\no,9]) = 0, n > no + 2, e = 1, 2, (4.17) 
where p{n,e) are given by (|4.ip - (|4.2p . 

Proof. We assume that the customers follow the (no,0)-mixed threshold strategy. Then the 
evolution of the system can be described by a Markov chain which is absorbed with probability 1 
in the positive recurrent closed class of states Sao ( \nQ, 9~\) = {{n, e) : < n < no+1, e = 1,2}. 
For the sake of brevity, we suppress the notation regarding [no, 6~\ in the rest of the proof . Thus, 
we will refer to the corresponding stationary probabilities Pao{n, e; [no, 9] ) by Pao{n, e). 

Since the Markov chain is finally absorbed in ^^'^([no,^]), we obtain immediately (|4.17p . 
The vector of the stationary probabilities {paoin^e) : (n, e) € Sad^ {\nQ,9'])) is obtained as the 
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unique positive normalized solution of the system of balance equations 

"0 + 1 

{Xl+ Hl + qi2)Pao{0,l) = q2lPao{0,2) + ^ fliPao{n,l), (4.18) 

n=0 

{Xi+ fJ-i+qi2)Pao{n,l) = q2iPao{n,2)+XiPao{n-l,l), l<n<no-l, (4.19) 

{\i0 + /Ui + qi2)Pao{no, 1) = q2iPao{no, 2) + XiPao{no - 1, 1), (4.20) 

(m + gl2)?'ao(^^o + 1, 1) = q2iPao{no + l,2) + Xi9pao{no,l), (4.21) 

no+l 

(A2 + /U2 + q2l)PaoiO, 2) = gi2Pao(0, 1) + X] ^2Pao(n, 2), (4.22) 

n=0 

(A2 + /i2 + g2l)?'ao(?^,2) = qi2Pao{n,l)+ X2Pao{n -1,2), l<n< no -I, (4.23) 
(A26' + /U2 + q2l)Pao{no, 2) = gi2Pao(?^0, 1) + X2Pao{no - 1, 2), (4.24) 

(Ai2 + g2l)?'ao(^^o + 1,2) = qi2Pao{no + 1,1) + X29pao{no, 2), (4.25) 

where we have included in ()4.18p and ()4.22p the pseudo-transitions from (0, e) to (0, e), e = 1, 2, 
with rate that correspond to visits of the facility at an empty system. 

For deducing the formulas (j4.14p - (j4.17p for the stationary probabilities, we may again follow 
the standard probability generating function approach, as it was briefly described in the proof 
of Proposition 14.11 However, given the formulas (j4.14p - ()4.17p . it is easy to check by direct sub- 
stitution that the stationary probabilities satisfy the equations (I4.18p -( j4.25p (using also simple 
summations for (|4.18p and (|4.22p ). ■ 

We can now readily conclude the following Corollaries 14.11 and 14.21 

Corollary 4.1 Consider the almost observable model of the stochastic clearing system in al- 
ternating environment, where the customers join the system according to the nQ-pure threshold 
strategy. The corresponding stationary distribution {pao{n,e; [?^ol)) is given by the formulas 

Pao{n,e;\nQ\) = p{n,e), < n < no - 1, e = l,2, (4.26) 

00 

Pao{nQ,e;\nQ\) = ^ p{n,e), e = 1,2, (4.27) 

n=no 

Paoin,e;\no]) = 0, n > no + 1, e = l,2, (4.28) 
where p{n,e) are given by (j4.ip - (j4.2p . 

Corollary 4.2 Consider the almost observable model of the stochastic clearing system in alter- 
nating environment, where the customers always balk. The corresponding stationary distribution 
{Pao{'n,e; [0])) is given by the formulas 

Pao(0,e;rOl) = pE{e), e = l,2, (4.29) 
Pao{n,e;\G\) = 0, n>l, e = l,2, (4.30) 

where Pe{s), e = 1,2 are given by (|3.7p - (|3.8p . 

We will now deduce the stationary distribution of the system when the customers follow an 
(no,^)-mixed reverse-threshold strategy. 
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Remark 4.1 Under an {nQ,9)-mixed reverse-threshold strategy with uq > 1, we have that the 
customers balk when they arrive at an empty system. Thus we have the stationary distribution 
of Corollary 

It is left to show what happens when the customers follow a (0, 0)-mixed reverse-threshold 
strategy. 

Proposition 4.3 Consider the almost observable model of the stochastic clearing system in al- 
ternating environment, where the customers join the system according to a {0,9)-mixed reverse- 
threshold strategy. For 6 = 0, the stationary distribution {pao{n,e; [0,0j)) is given by the for- 
mulas 

Pao(0,e; L0,0J) = pE{e), e = l,2, (4.31) 
Paoin, e;[0,0\) = 0, n>l, e = 1,2, (4.32) 

where PE{e^), e = 1,2 are given by (j3.7p - (j3.8p . 

For 9 G (0, 1), the stationary distribution [pao{n,e; \ 0,9\)) is given by the formulas 

oo 

Pao{0,e;[{),9\) = Y.{l-9Tp{n,e), e = l,2, (4.33) 

n=0 

oo 

Pao{n,e;[0,9\) = J^(l - 0)^->(i, e), n > 1, e=l,2, (4.34) 

i=n 

where p[n,e) are given by (j4.ip - (j4.2p . 

For 9 = 1, the stationary distribution {pao{n-, e; [0, Ij )) is given by the formula 

Pao{n,e;[0,l\) =p{n,e), n > 0, e = l,2, (4.35) 

where p{n,e) are given by (j4.ip - (j4.2p . 

The proof of Proposition 14.31 for 9 = is immediate, as in this case the customers balk 
whenever they arrive at an empty system. Therefore under such a strategy the corresponding 
continuous-time Markov chain is absorbed with probability 1 into the subset {(0, 1), (0, 2)} of the 
state space and the stationary distribution is the one given by (I4.29P and (|4.30p as in Corollary 
14. 2i In case 9 = 1, the customers always join so we apply Proposition 14.11 Thus, the only 
interesting case is for 9 G (0,1). Then, the proof of Proposition 14.31 follows a similar line of 
argument as the proofs of Propositions 14.11 and 14.21 Therefore, for the sake of brevity, it is 
omitted. 

4.2 Expected net benefit functions 

Based on the results of subsection 14.11 we can now compute the expected net benefit of a tagged 
customer if he decides to join the system after observing n customers upon arrival. Of course, 
his expected net benefit depends on the strategy followed by the other customers. Thus, we 
have various cases, according to whether the customers follow a threshold or a reverse-threshold 
strategy. We have the following Propositions I4.4ft4.6l and the Corollary 14.31 

Proposition 4.4 Consider the almost observable model of the stochastic clearing system in 
alternating environment, where all customers join the system. Then, the expected net benefit 
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Sao{n; [oo]) = Sao{n; [Oj) of an arriving customer, if he decides to join, given that he finds n 
customers in the system, is given by 

Sao{n;\(X)]) = Sao{n; 10\) = R-C ) ) n > 0, (4.36) 



1-Zi I \ 1-Z2 



where 



A = XiAiE[Si] + X2A2E[S2], (4.37) 

B = XiBiE[Si] + \2B2E[S2], (4.38) 

D = Ai^i + A2^2, (4.39) 

E = XiBi + X2B2 (4.40) 

and E[Si], E[S2], Ai, Bi, A2, B2, zi, Z2 are given by ([331) -(1331), g31)-(jMD and (Oil . 



Proof. The mean sojourn time of an arriving customer, if he decides to join, given that he 
finds n customers in the system is given by 

p-„(l|n; \oo])E[Si] + p~,{2\n; \oo])E[S2], (4.41) 

where Paoi^l^] [co] ), e = 1,2, is the probability that an arriving customer finds the environment 
at state e, given that he observes n customers in the system and that the [00] -strategy is followed 
by the other customers. The embedded (Palm) probabilities p~g{e\n\ \oo\) are given by 

where p{n, e) are given by (j4.ip - (j4.2p . Thus, the expected benefit of the tagged arriving customer, 
if he decides to join, is equal to 

Sao{n; M) = Sao{n; [Oj) = i?- Cfc(l|n; [00] )i^[5i] + p-„(2|n; fool )i?[52]]. (4.43) 

Plugging the formulas (jiT]) - ([0]) into (|4.42p and subsequently into (j4.43p yields (|4.36p . ■ 



Proposition 4.5 Consider the almost observable model of the stochastic clearing system in 
alternating environment, where the customers join the system according to the {no,9)-mixed 
threshold strategy. Then, the expected net benefit Sao{n; [no,^]) of an arriving customer, if he 
decides to join, given that he finds n customers in the system, is given by 



Saoin;\no,e]) = R-C- 



D 



< n < no — 1, 



Saoino;\no,9]) = R-C- 



(1 



\i-no 



Sao{no + l-\no,e]) = R-C- 



\i-nol 



E£„o+i[i-(i-^^""] 

where A, B, D, E, zi, Z2 are given by (j07P - (|0gp and (|^ . 



A 



r +5 T 



D 



l-z, 



-Z2 



(4.44) 



(4.45) 



,(4.46) 



13 



Proof. Assume that the customers join the system according to the (no, ^)-mixed threshold 
strategy. Then, the mean sojourn time of a tagged arriving customer, if he decides to join, given 
that he finds n customers in the system is given by 

p-,{l\n; \no,9])E[Si] + p~,{2\n; \no,e])E[S2], (4.47) 

where p~^{e\n; [no,^]), e = 1,2, is the probabihty that an arriving customer finds the environ- 
ment at state e, given that there are n customers in the system and that the [no, 0] -strategy is 
followed. The embedded (Palm) probabilities are given by 

XePao{n,e; [no, 6*]) 



Pao{e\n; \no,e]) 



1,2, 



(4.48) 



Xipaoin, 1; [no, 0] ) + X2Pao{n, 2; [no, 9] ) ' 

where Pao{n-,e; [no,^]) are given by (|4.14|) - (j4.16|) . Thus, the expected benefit of the tagged 
customer, if he decides to join, is equal to 

Sao{n; [no,^l) = i? - Cfc (l|n; [no, ^1 )i?[Si] + p-„(2|n; [no, ^1 )i^[S2]]. (4.49) 

Using the various forms of Pao{n, e; [no, 6*] ) in (|4.14p - (|4.16p yields ()4.44p - (j4.46p . ■ 



In the case of the no-pure threshold strategy, we obtain the following Corollarv 14.31 

Corollary 4.3 Consider the almost observable model of the stochastic clearing system in al- 
ternating environment, where the customers join the system according to the UQ-pure threshold 
strategy. Then, the expected net benefit Sao{n; [no]) of an arriving customer, if he decides to 
join, given that he finds n customers in the system, is given by 



A 



Sao{n\ [noD 



Sao{nQ] [no]) 



R-C- 



+ B 



1 



1-22 



D 



1-21 



E 



R-C- 



oo 

i=no 



A 



+ E 



1-21 



1 



< n < no — 1, 



1-22 

i 

+ B 



1-22 



E 



oo 

i=no 



D 



1-21 



+ E 



1-22 



(4.50) 



(4.51) 



where A, B, D, E, zi, are given by (I4.37j) - (|4.40j) and ([48I1 . 

Remark 4.2 Applying Corollary \4-3\ for uq = yields the expected net benefit Sao{0; [0]) = 
Sao{0', [ooj) of an arriving customer, if he decides to join, when the others follow the 'always 
balk' strategy. 

When the customers follow a (0, 0)-mixed reverse-threshold strategy, with 6 G (0, 1), we can 
use the same line of argument with Propositions 14.41 and 14.51 using the stationary distribution 
given by ()4.33p - ()4.34p . Then we have the following Proposition 14.61 



Proposition 4.6 Consider the almost observable model of the stochastic clearing system in al- 
ternating environment, where the customers join the system according to the (0, 9)-mixed reverse- 
threshold strategy for some 9 E (0, 1) . Then, the expected net benefit Saoin; [0, 9\ ) of an arriving 
costumer, if he decides to join, given that he finds n customers in the system, is given by 



Sao{n; LO,^J) 



R-C- 



EE,.(i - «)-" 






i 


E£„(i - «)-" 






i 



,n > 0, 



(4.52) 



where A, B, D, E, zi, z^ are given by (j071) - ([0Dl) and (gS]). 
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To express the various formulas reported in Propositions I4.4H4.5I and in Corollary 
expected net benefit function in a compact, unified way, we introduce the functions 



F{n,9) 
G{n,e) 



oo 



{RD - CA) 



l-zi 



+ {RE - CB) 



1 



Z2 



l-zi 



F{n,l) 
G(n,l) 



F(n,0) 
G(n,0) 



+ E 



, n > 0. 



Z2 



?i > 0, [0,1], 



Then we have 



Sao{n; [oo]) = Sao{n; [Oj 
Saoin; [no, 6*] 
Saoino; [no, 6*] 
(no + 1; [no, 6*] 
Sao{n; [no] 
Saoino; \no] 

SaoiO; m = Sao{0; [OOJ 

Saoin; [0,9\ 



F{n,l) 
G{n, 1) 
F{n,l) 
G(n,l) 

G{no,9y 

F(no,0)-F(no,g) 
G(no,0)-G(no,^)' 



H^{n), n > 0, 
iJ^(n), < n < no - 1, 



G(n,l) 

G(no,0) 

G(0,0) " 

F{n,9) 

G{n,9y 



= H'^in), < n < no - 1, 

- H^iO), 
n > 0. 



for the 

4.53) 

4.54) 
4.55) 



4.56) 
4.57) 
4.58) 
4.59) 
4.60) 
4.61) 
4.62) 
4.63) 



5 The almost observable case: Equilibrium strategies 

As we have already discussed in the beginning of Section lU it seems plausible that threshold 
strategies are adopted by the customers when the 'fast service' environmental state coincides 
with the 'less congested' environmental state, i.e. when (/ii — ^2){pi — P2) < 0. On the contrary, 
reverse-threshold strategies are plausible when the 'fast service' environmental state coincides 
with the 'more congested' environmental state, i.e. when the opposite inequality holds. This 
intuitive finding is associated with the monotonicity of H^{n) which plays a key role in the 
subsequent analysis. More specifically, we have the following Proposition 15.11 

Proposition 5.1 We have the following equivalences: 

H^{n) is strictly decreasing ^ AE - BD > <^ (;Ui - P2){pi - P2) < 0. (5.1) 
H^{n) is constant ^ AE - BD = Q ^ m = H2 or pi = p2. (5.2) 
(n) is strictly increasing 44> AE — BD < <^ {pi — P2){pi — P2) > 0. (5-3) 

The proof of this proposition is omitted, since its first case follows easily by simple algebraic 
manipulations that start from the relation H^(n + 1) — (n) < and lead to AE — BD > 
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and (^1 — lJ'2){pi — P2) < 0, through successive equivalences. The other two cases are treated 
similarly. Moreover, the monotonicity of the function with respect to 9 depends on the 

sign of (i^ii — P2){pi — P2)- Specifically, we have the following Proposition 15.21 

Proposition 5.2 We have the following equivalences: 
F(n, 9) 

is strictly increasing in 9 ^ AE — BD > 44> {fii — P2){pi — P2) < 0. (5.4) 



G{n,9) 
F(n,9) 



is constant in 9 ^ AE — BD = 4^ ni = 112 or pi = p2- (5-5) 



G{n,9) 
F(n, 9) 

is strictly decreasing in 9 ^ AE — BD < <^ (/ii — P2)(pi — P2) > 0. (5.6) 

G(n, 9) 

The proof of this proposition is also omitted, since the result is deduced easily after some algebra. 
We now state some properties of F{n,9), G{n,9) and H^{n), H^{n) that we will use in the 
sequel. Their proof is straightforward from their definition and thus it is omitted. 

Lemma 5.1 The functions F{n,9), G{n,9) satisfy the following properties: 

00 

F{n, 9) = ^(1 - 9y-''F{i, 1) = F{n, 1) + (1 - 9)F{n + 1,9), n > 0, 9 e [0,1], (5.7) 



i=n 

00 



G{n, 9) = ^(1 - 9y-'^G{i, 1) = G{n, 1) + (1 - 9)G{n + 1, 0), n > 0, ^ G [0, 1], (5.8) 

i=n 

G{n,9) > 0, n > 0, 6* G [0,1], (5.9) 
G{n, 9) is strictly decreasing with respect to 9 for any fixed n > 0. (5.10) 

Note that properties (15. 9p and (]5.10p of G{n, 9) assure that all denominators in (j4.56p - (|4.63p 
are positive. 

The intuitive discussion at the beginning of Section [4] in combination with Propositions 15.11 
and 15.21 suggests that we should methodologically proceed by considering separately three cases, 
corresponding to the sign (negative, positive or zero) of (/ii — H2)ipi — P2)- 

5.1 Case A: (yUi - H2){pi - P2) < 

In Case A, we will prove that an equilibrium threshold strategy always exists. Moreover, we 
will present a systematic procedure for determining all equilibrium threshold strategies. We first 
introduce several quantities that we will need in the sequel. 



Definition 5.1 Suppose that 
We define 



ifll - P2){P1 - P2) < 0- (5.11) 



nu = inf{n > 

hl = inf{n > 

n^ = inf{n > 

nf = inf{n > 



F(n,l)<0}, (5.12) 

F(n,0)<0}, (5.13) 

F(n,l)<0}, (5.14) 

F(n,0)<0}. (5.15) 
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Then, we have several properties of nu, nj;^, rijj and that we summarize in the following 
Lemma I5.2i 

Lemma 5.2 Suppose that (jS.lip holds. Then, there are three cases: 



Case I: F^(0) < 0. 
Then 



njj = = Ujj = n~j^ = 0, (5.16) 
F{n,9)<0, n>0, 9e[0,l], (5.17) 
F{n,0) - F{n,e) <0, n > 0, 9 e (0,1]. (5.18) 



Case II: i7^(0) > and liuin^oo (n) < 0. 
Then 



and 



1 <nu <oo, (5.19) 

F(n, 1)>0, 0<n<nu-2, (5.20) 

F(nc;-l,l) >0, (5.21) 

F(n, 1)<0, n>nu. (5.22) 



< ni < nc/, (5.23) 

F{n, 0) > 0, < 71 < nL - 1, (5.24) 

F{nL,0)<0, (5.25) 

F(n,0)<0, n>nL + l. (5.26) 



Moreover, 



UL, ifF{nL,0)<0 

UL + l, ifF{nL,0)=0, 

nu, if F{nu- 1,1) >0 

nu-1, if F{nu- 1,1) = 0. 



(5.27) 
(5.28) 



For every no G {n~^, . . . ,njj — l}, a unique solution 9 {uq) G (0,1) of the equation F{nQ, 9) 
exists with respect to 9, i.e. 



F{no, 9{no)) = 0, n+ < n < n" - 1. (5.29) 



Case III: limn-^oo H'^ (n) > 0. 
Then 



nu = riL = = n'^ = oo, (5.30) 
F(n,6i)>0, n>0, 6Ig[0, 1], (5.31) 
F(n,0) -F(n,6l) > 0, n > 0, 6Ig(0, 1]. (5.32) 
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Proof. In Case I, the condition H^{0) < in combination with the monotonicity of {n) 
(due to (j5.ip ) impHes that H^{n) < 0, n > 0. Then, we have that F{n, 1) < 0, n > and 
therefore F{n,0) = Y^Zni^ - Oy^''F{i,l) < 0, n > 0, 9 e [0,1] and F(n,0) - F{n,e) = 
E£n[l - (1 - Oy-^]F{i, 1) < 0, n > 0, G (0, 1]. 

In Case II, the conditions (0) > and liuin^oc (n) < 0, in combination with the 
condition (jS.ip for the monotonicity of H^{n) imply (j5.19p - (j5.22p . 

Equation K22t\ imphes that F{nu,0) = ESn^ -^(^' 1) < and we conclude KIM . Note 
also that by the definition oi we have immediately (|5.24p - (|5.25p . Moreover, we have that 
F(n,0) = X^i^„-^(^,1) < 0, for n > njj- For n with + 1 < ?i < n^/ — 1 we have also 
F(n,0) < 0. Indeed, suppose that there exists an n with til + 1 < n < nu — 1 such that 
i^(n, 0) > 0. Then we would have F{n — 1,0) = F[n — 1,1) + -F(n, 0) > and inductively we 
would obtain that F{nL,0) > 0, a contradiction because of ()5.25p . Thus F{n,0) < 0, for all 
n > riL + 1 and we obtain (|5.26p . 

Equations ([5:2711 and (|5:28]l are immediate from (IQOD - dO^I ) and ([521]) -([S^S]) respectively. 
Consider, now, an uq € • • • jn-^ — 1}. Then we have that > (since no < — 1 

- see (lOOl) ) and < (since uq > - see (lOGjl - dOTll ). By condition ([531), we have 

that is a strictly increasing and continuous function of 6, so by Bolzano's theorem we 

conclude that there exists a unique solution 0(no) G (0, 1) of the equation = 0. Thus, we 

obtain ([QO]) . 

In Case III, the condition lim.„^oo ('t-) ^ in combination with the condition (jS.ip 
for the monotonicity of (n) implies that (n) > 0, n > which gives F{n, 1) > 0, 
n > and F{n,e) = E»=„(l - Oy-''F{i,l) > 0, for n > and 9 G [0,1]. Moreover, 
F{n,0) - F{n,9) = E,=n[l - (1 - Oy-"^]F{i,l) > 0, n > 0, 61 G (0, 1]. Thus, we conclude 

Using Lemma 15.21 we will now prove the existence of threshold equilibrium strategies, when 
()5.1ip holds. We present the results in the following Theorem 15.11 

Theorem 5.1 In the almost observable model of the stochastic clearing system in alternating 
environment, where ()5.1ip holds, equilibrium threshold strategies always exist. In particular, in 
the three cases of Lemma \5.2 we have: 

Case I: (0) < 0. 

Then, there is a unique equilibrium threshold strategy, the \0~\-strategy (always to balk). 

Case II: H^{0) > and linin^oc (n) < 0. 

Then, an equilibrium pure threshold strategy always exists. Moreover, the equilibrium 
strategies within the class of all pure threshold strategies are the strategies [no] with uq = 
ri'LjiT'L + l,...,n[/. Also, the equilibrium strategies within the class of genuinely mixed 
threshold strategies are the strategies [no, 0(no)] with no G {n^, . . . , n^ — 1} and 9{nQ) the 
unique solution in (0,1) of F{no,9) = with respect to 9. 

Case III: liuin-yoo (n) > 0. 

Then, there is a unique equilibrium threshold strategy, the \oo~\ -strategy (always to join). 

Proof. Case I: Consider a tagged customer at his arrival instant and assume that all other 
customers follow an [no] strategy for some no > 0. Inequality (I5.17P and relations ()4.60p and 
(|4.6ip imply that the expected net benefit of the tagged customer, when he finds n customers 
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and decides to join is Saoi^n] [rag]) < 0, for < tt. < hq. Thus, he always prefers to balk and his 
best response against [no] is [0] . 

We now assume that all other customers follow an [no,^ol strategy, for some Uq > and 
^0 £ (0, !)• Then, if the tagged customer finds n customers at his arrival instant and decides to 
join, his expected net benefit will be Saoin-', In-o, Oq]) < for < ?i < no + 1, from (j5.17p - (|5.18p 
and (|4.57p - (|4.59p . Therefore, the tagged customer is always unwilling to join and we have that 
his best response against [no,^o] is [0]. 

If all customers follow the [oo] strategy, (|5.17p and (j4.56p yield Saoin; [oo]) < for n > 0. 
Again, due to the negative expected net benefit, it is preferable for the tagged customer to balk. 
So, his best response against [oo] is [0] . Thus, we conclude that the only best response against 
itself within the class of (pure and mixed) threshold strategies is [0] . 

Case II: Consider a tagged arriving customer and suppose that all other customers follow 
an [no] strategy, for some < til — 1. If the tagged customer finds no customers and decides 
to join, his expected net benefit will be Sao{nQ; [no]) > 0, from (j5.24p and (|4.6ip . This implies 
that when he finds no customers, he is willing to join. Thus, [no] cannot be a best response 
against itself. So such a strategy cannot be an equilibrium. 

Consider, now, a tagged arriving customer and suppose that all other customers follow an 
[no] strategy, for some uq > nu + 1- Using (j4.60p and (j5.22p . we have that Saoin; [no]) < 0, 
for njj < n < no — 1. This means that when the tagged customer finds n customers, with nu < 
n < no — 1, then he is unwilling to enter. Thus, the [no] strategy cannot be an equilibrium. We 
conclude that the search for equilibrium strategies within the class of pure threshold strategies 
should be restricted to strategies [no] with ni < < nu . 

We mark an arriving customer and we assume that all other customers follow an [no] strat- 
egy, for some no with n^ < no < nu- Prom (Onil . (ITHTD . (lOnD . (I5:2TD . (1051) and (lOHIl . 
we have that the expected net benefit of a customer who finds n customers upon arrival and 
decides to join is 5ao(n; [no]) > 0, for < n < no — 1 and Sao{no; [no]) < 0. Thus [no] is a 
best response against itself and we conclude that all such strategies are equilibrium strategies. 

To finish with our search for equilibrium strategies in the class of pure threshold strategies, 
we have to examine the [oo] strategy. This cannot be an equilibrium, since (j4.56p and (j5.22p 
imply that Sao{n; [oo]) < 0, for n > njj, which means that it is not optimal for the tagged 
customer to join when he sees n customers for some n > nu. Therefore, we conclude that the 
equilibrium strategies within the class of pure threshold strategies are exactly the strategies [no] 
with riL < no < njj. 

We will now search for equilibrium strategies in the class of genuinely mixed threshold 
strategies, i.e. among strategies [no,0o] with ^o £ (0, !)• A mixed threshold strategy [no,^o] is 
an equilibrium if and only if the relations F{n, 1) > 0, for < ??, < no — 1, F(no,6Q) = and 
F(no, 0) — F(no, ^o) ^ hold (see (I4.57|) - (I4.59|) ). A moment of refiection shows that [no, ^o] may 
be an equilibrium only if [no] is an equilibrium (see (|4.57p - (|4.59p in comparison with (j4.60p - 
()4.6ip ). Thus, we should restrict our search for equilibrium genuinely mixed threshold strategies 
to strategies [no, ^o] with no = n^, n^ + 1, . . . , nu. 

If F(nL,0) = 0, then there does not exist 9 G (0,1) such that F{nL,0) = 0, since 
is strictly decreasing. Therefore, [nL,0] cannot be an equilibrium strategy for any 6 G (0,1). 
Similarly, if F{nu — 1,1) = 0, then the strategy \nu — 1, 0~\ cannot be an equilibrium strategy for 
any 9 G (0,1). Moreover, \nu,0~\ cannot be equilibrium for any 9 G (0,1), since F{nu,0) < 0, 
6 G (0,1). Therefore, a strategy [no,^o] with 9 G (0,1) may be an equilibrium only if n^ < 
no < n^ - 1. 
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Now, for every uq € {nj, . . . , — 1}, the only [no, ^ol strategy that can be an equihbrium 
is the one that corresponds to 60 = ^(^o), since F{no,6{no)) = 0. Indeed, if all customers 
follow the [no,^(no)] strategy, the expected net benefit for a tagged customer, who finds n 
other customers and decides to join the system, is Sao{n; [no,^(rao)]) > 0, for < n < no — 1, 
Sao{no; [no,6'(no)l) = and Sao{no + 1; [no,6'(no)l) < 0, from ([CT]) and (fCTj) . Thus, 

[no,0(no)] is an equilibrium strategy. 

Case III: Following the same line of argument as in case I, we now find that when all cus- 
tomers follow a pure threshold strategy [no] or a mixed threshold strategy [no, ^ol the expected 
net benefit function is always positive. Thus, the best response of a customer is always to join 
the system. Thus, the only best response against itself in the class of threshold strategies is the 
[00] strategy. ■ 

Note that although pure threshold strategies always exist, it is possible that genuinely mixed 
threshold strategies do not. This happens if n^ — 1 < n^. 



5.2 Case B: (/xi - /X2)(pi - P2) > 



In Case B, we seek for equilibrium strategies in the class of reverse-threshold strategies. We 
will exclude strategies [no J and [no,^oJ with no > 1. Indeed, all these strategies prescribe to 
balk, when a tagged arriving customer sees an empty system. Thus, under such a strategy, the 
system remains continuously empty, after the first service completion. Therefore, in steady state, 
these strategies are equivalent to the 'always balk' strategy [ooj . Thus, we seek for equilibrium 
strategies only in the set Sr-t = {[Oj, [ooJ} U {[O,0oJ '■ ^0 £ (0)1)}- We first introduce several 
quantities that we will use in the sequel. 



Definition 5.2 Suppose that 
We define 



-^2)(pi - P2) > 0. 



mu 
rriL 



m 



u 



m 



inf{n > 
inf{77, > 
inf{n > 
inf{?7, > 



F(n,l) >0}, 
F(n,0) > 0}, 
F(n,l) >0}, 
F(n,0) > 0}. 



(5.33) 

(5.34) 
(5.35) 
(5.36) 
(5.37) 



Then, we have several properties of mu, mi, rrijj and that we summarize in the following 
Lemma [ 



Lemma 5.3 Suppose that (|5.33p holds. Then, there are three cases: 

Case I: {0) > 0. 

Then 



mu = mi 



mr 



F(n,6')>0, n>0, 6'g[0,1]. 



(5.38) 
(5.39) 
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Case II: H^{0) < and limn^oo H'^ (n) > 0. 
Then 



and 



Moreover, 



1 < mu < oo, (5.40) 

F(n,l)<0, 0<n<mu-'2, (5.41) 

F{mu- 1,1) <0, (5.42) 

F(n, 1)>0, n>mu. (5.43) 



< mi, < mc/, (5.44) 

F(n, 0) < 0, < n < rriL - 1, (5.45) 

i^(mL,0)>0, (5.46) 

F(n, 0) > 0, n > mi + 1. (5.47) 



I mi + l, ^/F(mi,0)=0, ^^"^^^ 

^ [mu-1, if F{mu - 1,1) = 0. 

If m'^ = and > 1, then there exists a unique 9{0) G (0, 1) such that 

F{0,e{0)) = 0, (5.50) 

F{n,9{0)) > 0, n > 1. (5.51) 



Case III: lim.„_ 
Then 



,H^in) < 0. 



mu = mi 



m. 



mi 



oo. 



F{n,e) < 0, n > 0, G [0,1]. 



(5.52) 
(5.53) 



We omit the proof of Lemma 15.31 as it is completely analogous to the proof of Lemma 15. 2i 
We are now in position to prove the existence and uniqueness of reverse-threshold strategies, 
when (|5.33p holds. We present the results in the following Theorem 15.21 The statements about 
the uniqueness of the reverse-threshold equilibrium strategies should be interpreted within the 
class Sr-t = {[OJ; L°oJ} U {[Oi^oJ : G (0, 1)} of reverse-threshold strategies. 

Theorem 5.2 In the almost observable model of the stochastic clearing system in alternating 
environment, where (j5.33p holds, equilibrium reverse-threshold strategies always exist. In par- 
ticular, in the three cases of Lemma 15.31 we have: 

Case I: (0) > 0. 

Then, there is a unique equilibrium reverse-threshold strategy, the [Oj strategy ('always to 
join'). 
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Case 2. H^{0) < and liuin^^ {n) > 0. 

// = 0, the [Oj strategy ('always to join') is the unique equilibrium reverse-threshold 
strategy. If > I, then the [ooj strategy ('always to balk') is the unique equilib- 
rium reverse-threshold strategy. Otherwise, the LO>^(0)J strategy is the unique equilibrium 
reverse-threshold strategy. 

Case III: lim.„^oo -f^^(n) < 0. 

Then, there is a unique equilibrium reverse-threshold strategy, the [ooJ strategy ('always 
to balk'). 

Proof. Case I: Consider a tagged customer at his arrival instant and assume that ah other 
customers follow the [Oj strategy. Inequality (|5.39p and relation (|4.56p imply that his expected 
net benefit, when he finds n customers and decides to join is Saoin; [Oj) > 0, for n > 0. Thus, 
he always prefers to join so his best response against [Oj is [Oj itself. 

Similarly, let mark an arriving customer and suppose that all other customers follow a [0, ^oj 
strategy, for some S (0, 1). Then, the expected net benefit of the tagged customer, who finds 
n customers at his arrival instant and decides to join, will be Saoin', [0, 9q\) > 0, for n > due 
to ()5.39p and (j4.63p . Therefore, the tagged customer is always willing to join and we have that 
[Oj is the best response against [0, ^oj • 

If all customers follow the [ooJ strategy, equations (|5.39p and (|4.62p imply that 5ao(0; [ooJ ) > 
0, so the tagged customer prefers to join. Thus, we have again that [Oj is the best response 
against [ooJ . So the only reverse-threshold strategy which is best response against itself is the 
[Oj strategy. 

Case II: Assume that m^ = 0. Then F{0, 1) = and mu = 1. Consider now a tagged 
customer at his arrival instant and suppose that all other customers follow the [Oj strategy. In- 
equality (I5.43P and relation (I4.56P imply that his expected net benefit, when he finds n customers 
and decides to join is Saoin; [Oj) > 0, for n > 0. Thus, [Oj is a best response to itself. 

Assume, now, that > 1, which means that F(0,0) < 0. If we consider a tagged arriving 
customer and suppose that all other customers follow the [ooJ strategy, then the tagged cus- 
tomer, if he finds customers and decides to join, has expected net benefit S'ao(0; Yoo\) < 0, due 
to (|4.62p . Thus the strategy [ooJ is best response against itself, i.e. it is equilibrium strategy. 
Otherwise, we will have = 0. Consider again a tagged customer and suppose that the other 
customers follow the [O,0(O)J strategy. Then, if the tagged customer finds n customers at his 
arrival instant and decides to join, his expected net benefit will be either Saoi^\ V^-, ^(0)J ) = if 
n = 0, or Saoin] [0, 6'(0)J) > 0, if n > 1, due to (f530D - (l53T]l and gJ3|). Therefore, the [0, 6l(0)J 
strategy is equilibrium strategy. 

Case III: Following the same line of argument as in case I, we now conclude that the expected 
net benefit function is negative. Thus, the best response to every reverse-threshold strategy is 
[ooJ . Thus the only equilibrium reverse-threshold strategy is [ooJ . ■ 

5.3 Case C: (yUi - yU,2)(pi - P2) = 

Case C occurs when ui = uo or — = — . In this case, the distinction 'fast environmental state' 
and 'slow environmental state' has no sense or the distinction 'more congested environmental 
state' and 'less congested environmental state' has no sense. Therefore, we conclude that the 
information on the number of customers in the system, does not affect the decision of a tagged 
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arriving customer. A similar analysis is possible as in the other two cases and we have the 
following Theorem I5.3i 

Theorem 5.3 In the almost observable model of the stochastic clearing system in alternating 
environment, where 

fJ-i = /W2 or pi = p2, (5.54) 

an equilibrium strategy exists within the class of threshold and reverse-threshold strategies. In 
particular we have the following three cases: 

Case I: (0) < 0. 

Then, the unique equilibrium strategy in the class of threshold and reverse-threshold strate- 
gies is the [0] = [ooj strategy ('always to balk'). 

Case II: if^(O) = 0. 

Then, every strategy in the class of threshold and reverse-threshold strategies is equilibrium 
strategy. 

Case III: H^{0) > 0. 

Then, the unique equilibrium strategy in the class of threshold and reverse-threshold strate- 
gies is the [oo] = [Oj strategy ('always to join'). 

6 Summary and conclusions 

In this paper we considered the problem of analyzing customer strategic behavior, in a clearing 
system in alternating environment, where customers decide whether to join the system or balk 
upon arrival. We identified four cases with respect to the level of information provided to arriv- 
ing customers and derived the equilibrium strategies for each case. It is important to notice that 
in each case we identified all equilibrium strategies within the appropriate class of strategies. 
Moreover, in the almost observable case, which is the most interesting one. Theorems 15.11 15.21 
and 15.31 suggest that the equilibrium strategies in the class of threshold and reverse-threshold 
strategies are completely characterized by the signs of the quantities {pi — ^2) {pi — P2), {n), 
Y\mn^ooH^{n) and H^{n). Thus, we can easily combine these theorems and develop an algo- 
rithm for determining the equilibrium strategies. We present the algorithm in pseudo-code form 
in Figure 1. Figure 2 shows schematically the various cases I,II,III when (/ii —^2) {pi — P2) < 0. 

We have also to notice that the results in the almost observable case are qualitatively dif- 
ferent for the two cases A and B, where (^1 — /i2)(pi — P2) is negative and positive respectively. 
Indeed, in case A, there is, in general an interval of thresholds that constitute equilibrium 
threshold strategies. On the contrary, in case B, there is a unique equilibrium reverse-threshold 
strategy. These observations correspond to the regimes of Follow-The-Crowd (FTC) and Avoid- 
The-Crowd (ATC) as defined in Hassin and Haviv (1997, 2003). Indeed, in case A, where 
(/ii — /i2)(/0i — P2) < we have that the 'fast service' environmental state coincides with the 
'less congested' environmental state. Then, we can argue as follows, if we want to compare two 
threshold strategies with thresholds n and n + 1: If the customers follow a threshold strategy 
with threshold n and an arriving customer observes n customers in the system, then he de- 
duces that at least n customers arrived since the last clearing epoch. If the customers follow a 
threshold strategy with threshold n + 1 and the arriving customer observes n customers, then he 
deduces that exactly n customers arrived since the last clearing epoch. Thus, in the latter case, 
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the arriving customer has the sense that the system is less congested and therefore the environ- 
mental state is most probably the 'fast service' one. We conclude that the arriving customer is 
more willing to enter the system. Therefore, if the customers adopt a higher threshold, an arriv- 
ing customer tends to follow them in adopting a higher threshold and we have an FTC situation. 

On the other hand, in case B, where (/xi — H2){pi — P2) > 0, we have that the 'low service' 
environmental state coincides with the 'less congested' environmental state. The usual defini- 
tion of the ATC situation is not applicable here, since we consider reverse-threshold instead of 
threshold strategies. Moreover, under any reverse-threshold strategy [n,6\ with n>l, the sys- 
tem remains continuously empty after the first visit of the transportation facility and so we have 
excluded these strategies in our seek for equilibrium strategies. Thus, in case B, we will limit 
our intuitive discussion of the ATC phenomenon to the class of strategies {[O,0J : ^ € [0, 1]}, 
as we have already done in the analysis of subsection 15.21 Suppose that the customers follow 
a reverse-threshold strategy [n, 9\ and then they move to another reverse-threshold strategy 
[n, 9'\ with 0' > 9. Consider now an arriving customer that finds customers in the system. 
Knowing the strategies of the other customers, the arriving customer has the sense that the 
system is in the less congested environmental state in the second case, where the customers 
enter with probability 9'. Indeed in this case, the customers are more willing to join than in the 
first case (since 9' > 9) so the information of an empty system imply that it is more probable 
that the system is in the less congested environmental state. Therefore, the customer becomes 
less willing to enter, as the less congested environmental state coincides with the low service 
state. Thus, when the other customers increase the probability of entering, the tagged customer 
tends to decrease his probability of entering, i.e. we have an ATC situation. 

The focus of this work was on equilibrium analysis. On the other hand, one can think 
of a situation where a central planner employs acceptance policies that maximize the social 
benefit, under the various levels of information on the system state. It is easy to see that in the 
fully unobservable, the fully observable and the almost unobservable cases the strategies that 
maximize the social benefit are the equilibrium strategies. This coincidence between equilibrium 
and socially optimal strategies can be explained by the total removals. Since the server removes 
all customers at service completion epochs, each customer who decides to join does not impose 
any externalities to other customers. In the almost observable case equilibrium and socially 
optimal strategies are identical except from the case where {n) is strictly decreasing, (0) > 
and lim„^oo {n) < 0. In this case the unique socially optimal strategy is the \nu] strategy, 
which is also an equilibrium. 
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ALGORITHM 

IF (^1 - ^t2)(pi - P2) < THEN 

IF H'^{0) < THEN "Equilibrium threshold strategies: [0]." 

ELSEiF linin^oo H'^ (n) > then "Equilibrium threshold strategies: [oo]." 

ELSE 

% Compute nu ■ nu = inf{7i > : F{n, 1) < 0} 
nu = 

while F{nu, 1) > DO 

nu =nu + 1 
endwhile 

% Compute riL ■■ ul = mf{n > : F{n, 0) < 0} 
riL = nu 

while F{nL — 1, 0) < DO 
riL = riL - I 

ENDWHILE 

% Compute : n'^ = inf{7i > : F{n, 1) < 0} 
IF F(nu — 1, 1) > then 

— nu 

ELSE 

n^ — nu — 1 

ENDIF 

% Compute n+ : n+ = inf {n > : F{n, 0) < 0} 

IF F{nL,0) < THEN 

— nu 

ELSE 

= + 1 

ENDIF 

"Equilibrium THRESHOLD STRATEGIES: \no], no = nu, nu + 1, nu" 

IF TlJ < — 1 THEN 

+ — -I 

FOR no = nj^ : n^j — i 

Compute 6{no) : F{no,0(no)) = 

ENDFOR 

"Equilibrium mixed threshold strategies: \no,6{no)], m e {nj, n|} — 1}. 

ENDIF 
ENDIF 

ELSEIF (/ii - fl2)ipi - P2) > THEN 

IF H'^{0) > THEN "Equilibrium reverse-threshold strategies: [OJ." 

ELSEIF limn^ca H'^ (n) < then "Equilibrium reverse-threshold strategies: [ooJ." 

ELSE 

IF _F(0, 1) = then "Equilibrium reverse-threshold strategies: [Oj." 
ELSEIF F(0, 0) < then "Equilibrium reverse- threshold strategies: [coJ." 

ELSE 

Compute 61(0) : F(0,6i(0)) = 

"Equilibrium reverse-threshold strategies: [0,6'(0)J." 

ENDIF 
ENDIF 

ELSE 

IF H'^{0) > THEN "Equilibrium threshold strategies: [0]." 

ELSEIF H'^{0) < THEN "EQUILIBRIUM THRESHOLD STRATEGIES: [co]." 

ELSE "Equilibrium threshold strategies: [n] , n > 0." 

ENDIF 



Figure 1: Computation of equilibrium threshold/reverse-threshold strategies 
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Case I 



Case II 
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Unique equilibrium: 
join ([oo]) 



Figure 2: Case A - {fii — ^2){pi — P2) < - Equilibrium threshold strategies 
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